So we are here to just make the point. Well make the point, yeah. Make the status about the shot detection. To so you have applied the shot detector to some of the CINETIS video, or at least one, and uh so we can basically maybe just comment on on this, maybe quickly remind what the shot detector is how the shot detector is working, basically. Uh we might be able anyway to give you some reference. I mean your report it's it should be described there or yeah, yeah. You have the report? Okay. To Olivier, yeah. Ah, okay, yes, the kayak wi I was not even with the other one, okay. By the way, I was uh going to ask, so I had seen the video, i you uh did um a downsampling of the video, right? I mean For uh browsing I mean. Okay, yeah. So this is hierarchical normally, so you have scenes and shots, but Because for instance we could probably, you know, like group all the initial ones into one scene and maybe after, when there's some things inside, it s should be another scene, so. Which are extracted I think uh Downsampling. Yeah. Yeah, the quality is that's why I was asking that. It it's not this that you processed, right? Okay, okay. So here we see that we the shot at eighteenth and after twenty twos and after again. And uh if we go down, I mean maybe in terms of shot here, we can see that Right, I mean so now we see shot seven I suppose, yeah. Uh yeah, okay, shot nine currently, shot ten probably. Eleven or twelve. No, eleven. Or no this is ten indeed, okay. Yeah. Yeah, this one, it's eleven, twelve or fourteen seconds. Yeah, okay. Okay. Yeah, it's inside and yeah. It's really a dissolve or is it really like is you have the feeling that it's both Okay. But it's it's it's not standard man-made movies, because I would say most of the time people couldn didn't do any dissolve. Yeah, I could see here, I mean, that it was like horse it was only one single one. N yeah, yeah. This is the one I I had seen, yeah. But as I said, I mean uh with even the motion detector, when it's black it's there is not s no structure, so you don't esti yeah, but you don't really estimate motion. Well, there is no structure. There is no texture, if you want to say so, and this is this is not so good for the motion. Uh but for instance of course between the horse and uh and um the c the c uh uh yeah. Uh the cows, yeah, maybe we could have detected things. But it's not even sure, because the background is quite similar and uh it's it's it's not that sure. So it's yeah. The histogram, so we have experience with it. So it's it's scrambled, so you can scramble the image and you get the same histogram. So there is no structure indeed, so of course when you look the horse and then you can see as a cow the cows, of course you have a black in the top and uh you have also uh maybe eighty percent of like green uh, which is even not green there but uh uh grass. You have stones and uh so there of course are differences, because on the one hand you have horse, but it's black, so you have more sky, which is black on the right, so you don't care about this. So it's a glo it's global it's a global histogram, so yeah. But it's when for this I would for this application I could imagine that it would were quite high. Uh and it's a threshold but still it's adaptive, right, if I remember well, yeah. So. Because of course if you just compute the yeah, if you have the report because if you compute the um distance between the two histograms, I think this was Bhattacharya? Right. So now we know. And uh so if you take uh these two with the with the Bhattacharya uh then of course you should just do a threshold. When there is motion you will see a lot of things changing, so by using an L_R_T_T_ threshold, if you see that there is a continuous change of the d distance, you don't do a threshold. It's only if you have an identified peak, which is m fifty percent or maybe uh two times higher than the the next second value in the window, then you say it's it's a shot. Right, I don't know. Yeah, yeah. Yeah. It's continuous for dissolve, yeah. But an anyway it's difficult, I mean in th in in in general, because sometimes you have some dissolve you could imagine they are part of of of a scene and uh so, but of course when you have cuts you can see they are easy to detect, right, I mean like something yeah. So uh yeah, so with the motion basically the motion tried to estimate well the motion between two frames, right, if my image is there but after it's there, and it looks at how many points c are well matched. So you can see that like somebody speaking and so and so before the sh before the shot you can see that usually there are like ninety percent of the points at least, right, who have some motion, so uh But then after suddenly if you go from one frame and you try to match all the points here with like a translation, plus the zoom plus maybe some rotation, you c you can find anything well, right, so basically you would see that the number of points maybe only ten percent of the points you can match well, so it's easy to detect. And uh after you can see that indeed because it's a long shot and it's almost like a flat plane, you can do a very good registration through the number of points. Is very high, I mean the number of points that are matching. Uh and here you basically here with a histogram uh you can see it very flat, but of course you should go from this histogram to this which is essentially green. You have this big uh measure, which is here uh the chi two I don't know whether this is what you implement is used in the no it's It's Bhattacharya. Okay. Yeah, so yeah, and you have maybe a one image with a um adaptive threshold, I think. You had something like this, right? Just to explain uh how Yeah, uh Yeah, you see like here. Yeah. Alpha times the maximum of the other ones. Of the other ones. So the of course if it's you are not a maximum, you are not going i inside the window you are not the max, anyway you are not going to be to be above uh alpha times the max of the others, right, uh because alpha is low lower than one. Uh. Like for instance yeah. Like for instance here I I see six, but I don't know whether this was for Bhattacharya or for uh chi s chi two. Yeah. Yeah, yeah, yeah. Two point four, yeah, so it's a bit less because the amplitude is much smaller, yeah. uh yeah. So you can uh it was not so just I mean because the motion I mean it was uh in the shot we've seen, f because f especially with chi two square it r the amplitude it's much bigger than with Bhattacharya, right. Uh you can see that it could go to seven hundred, and when it's there is nothing, it's can be zero. With Bhattacharya it's between zero and one, so you have not so much amplitude. Um but uh still uh in some of the v these videos for instance even if we gon go now if we want to, but now for the kayak case, I don't know. Uh at some point targeting two kayaks and it's moving quite fast towards to other people. I don't know what happened there, if it was detected as a shot transition or Yeah, yeah. It was it as a shot, okay. I was uh wondering yeah yeah, maybe. But here for instance you understand so we can show this tomorrow to uh to the people at the ci CINETIS meeting, I mean to Right. Yeah, okay, yeah, we need a connection, that's true. Yeah, that's true. Yeah, I can bring my laptop if necessary, yeah, yeah. I'll I'll bring it, okay, so it's it's done. Well yeah, but normally there is a a burning if you c maybe you can play the shot or It's strange I mean, maybe you play yeah, if you play the the video well, insi at the beginning, I don't know what's happening. Ah, you didn't generate the okay, the Real Media. Okay. Well no, I mean here I can see that it was it was detected as a single shot. Okay. I was because here you can see that the motion uh here is uh, you know, he is uh doing a zoom on the two kayaks here. Yeah, and then he's doing quite a fast transition from these to another one, and I was wondering whether so if you well though of course it's true that it's mainly water, so maybe the colour is indeed uh quite but you know, as a kayak is entering quite fast, I mean you can have quite maybe good distance. But I see that it seems to be how many shots do you have indeed de de detected? Six, yeah, the one is uh maybe so you did it with with what by the way? It's with the images I gave you or okay. Ah, to regenerate an A_V_I_ file, okay, yeah, yeah. Okay. Okay, yeah. Yeah. Yeah, I mean initially I don't know why there is a a shot detected. Ah okay, so the okay so the here is an image number. In your programme. Ah okay, okay, yeah. Okay, okay so then this So, but it's in seconds or it's Okay. Okay, yeah, okay, okay, seconds plus number. Okay, so basically what has been uh yeah, extracted is yeah, this point rather than okay, so this is normal, okay. Okay. Okay, like transition, something that is okay. Yeah, it's motion yeah, it's not a cut. Yeah, but I was thinking that maybe it could have been. But it's true that it's mainly water when we see things, so it's, yeah, it's not such a big uh but yeah, sometimes I mean you can have various things hap I mean just think about more action movie and thing like this. Well sometimes it's very difficult, so focus, usually there is no problem, people say this is a cut, this is not a cut, but like for dissolve even sometimes, you know, people may it's people may have some trouble I mean this is a dissolve this is not this should this should be labelled as a transition or not. Some people will say yeah, but it's part of the thing, or uh for instance in like in um also in uh, come on, uh advertising usually. Because the shots are very short in advertising, usually it's uh very fast and so sometimes, yeah, when you have an incrustation like last time, what what should you say, it's it's, yeah, it's a change of But uh Yeah, I mean it's a so sometimes very difficult. A at least what I can see that currently like for CINETIS uh for instance if it's to do colour correction or for instance if it's too black, too a bit whiter, I think this type of things should be fine. Because basically it's even if they are not real shots, they are not so interested if it's not real shot. Maybe after they have this and they may say okay, just by looking at least the summary, so maybe for the summary we could do better uh key frame extraction for instance, but by just looking at the key frames and so on they may say okay, from this shot to this shot we apply like uh specific um colour enhancement algorithm or restoration algorithm or uh to improve uh the brightness and something like this right. And uh so I think for them this is fine um after i yeah. Well, maybe after I mean they would sh okay, uh they would go directly and say select the shots from here to here, and they would say you apply this algorithm, yeah. So of course we could try to make scene clustering and so on, but I'm not even sure that this is necessary for them at this point, because uh if they have a six minute movie, if they have thirty shots, for them it might be easier to look at from here to here I apply this, and they can do it quite quickly. And they have to do it an to do it anyway, because what's going to take time might be uh to select the type of restoration algorithm or the type of things. So this might take more time than having to click two times on two shots rather tha tha than uh having directly a scene, right? So so I think for them uh this type of things is going to be useful uh, at least for this task. If they want to do stabilisation, it may not be sure of that. Uh this is the right segmentation for some of these, but so but this is another point which for motion stabilisation, as I said several time, we are not we did not commit to provide anything, but of course if we can provide things, we'll do it, but uh yeah. So do you have uh more question or uh do you have a Well I just i i i it's I think this is Yeah, but after I mean it may be more like a guy um i int i in an interface, a guy interface. I mean where I mean if second select from shot one to seven, uh this is a same seg they may do the grouping by themselves quite by looking at this it may take w w less one one minutes. And I think it's anyway, if it's too summarised after they may want to go inside uh to see whether there is not special things I mean, so. Uh because after it depends, I mean if you give only one key frame per shot, after I mean it's one hundred seventy images, it's quite fast, right, and so I think and the hierarchical thing is good, because still you can check if you have some ambiguities you are not sure, you could Yeah. Yeah. Yeah, at least yeah, yeah, yeah. Yeah, yeah, yeah. Well, ultim ultimately I think this is the type of things, but at this point we uh yeah. I I hope uh that uh if it's a big success uh they will ask IDIAP to to provide something like this of course with some money or at least to pay a student or somebody for doing it, but uh yeah, of course this was ultimately I think one way of uh if you could provide like chapters or something like this, indeed you could provide like the A_V_I_ to the clients, and he may ask for special services or uh and even to just to watch the movie, to have direct access to different parts and so on. But it's uh I mean uh this labelling might be more easy to do for the client by the client, because he knows what they're I mean he we wants to have this and this and uh right, so. Yeah, yeah, he may want to organi yeah, yeah. Otherwise, I mean this but of but of course it might be easy to do a very simple software, which would allow the user to select just the shots and give names, and after it would generate automatically some index and uh as a I suppose the client might be interested in such a such a things if he could uh generate automatically on the D_V_D_. But for and you could imagine that before sending in the D_V_D_, you could allow him to l to watch this type of interface, and he would be able to do the selection, say this is this, this is this, this is this, this is this, and when he would receive the D_V_D_, he would indeed uh get already the chapters that he decided to put there. Yeah, yeah, yeah. And I think this may it it's good that this is a demo like this, because I'm sure that they might be interested indeed, because I imagine that some people Yeah. Yeah, yeah, yeah. Uh yeah, so I think this is uh this is a very good uh and a yeah. But uh So this is a kind of service, I don't know, either they give it for free and uh But uh But you see that sometimes even here it's a bit like uh it's for instance if people would like to have like some motion compensation, uh you i it could be proposed. But for instance here it I wou it would be problematic to have motion. So i here it's fine to do motion compensation, but stabilisation rather than yeah. But For instance here it's it's it's it's Effects? Okay. Ah okay, so maybe like here for instance it is a just reproduced two times uh. Yeah. So this is vi okay. This video is about four minutes, five minutes, right? Ah okay, so this is not the one that has been uh Okay. Okay. It's it's it's real time, right? I mean the processing. Yeah. A bit more than real time? Well anyway, yeah. Yeah, normally it could it could go faster. But i probably it's probable that even for the project you would do some of the reprogramming, I mean to especially if uh we need to interface new uh um yeah, or for just reading, I mean the the D_V_X_ and thing like this, so it's uh uh so tomor anyway for this we don't know exactly yet, because it's not sure what they want to what the user's tools for reading D_V_D_ and so on, so. And s because I don't think this is so difficult, now you have um already looked at the histograms and open C_V_ computing distances and so on. So if we want to do a special thing that probably runs fast, it may be easier to do it directly and uh yeah. But uh but yeah, the interface is very so the interface uses the output of your programme. Essentially, okay. Okay. Mm. You can do the get frame. But otherwise it's not necessary. Oh yeah, no no no. Ye okay. Oh no no, you need you need to generate these images, right? Okay. Okay. Okay, yeah. Yeah. As a directory, yeah, yeah. Yeah. Yeah, yeah. Yeah. X_M_L_, yeah. On the shot. And it's a simple format I suppose, like uh like you have scenes and shots and uh key s some time stamps. Yeah. So but you're yeah. Yeah, I mean it it might be good to And who has a right to put who has a right to put data on the M_M_M_? Okay. Okay, yeah. Yes. Ah, okay, yeah. Yeah, if yo If you have the same uh decomposition right? Yeah. Yeah, but it may happen that uh for ninety fra ninety percent of the frame it's different, so you see it fine, but suddenly you have one other frame, they don't know what it is and where it comes from. Yeah, yeah. Yeah, yeah. Oh, you mean bac because it's it's it's the M_M_M_ cache? Okay. So this means that when it's done, if I go on my computer and I do the same, it's not going to enter in my cache, but it's going to be already there, in M_M_M_, okay. So it's th it's so it's not done on it's so it's not done online. The first time, yeah, but after it's it take okay. Okay, okay. I thought it was everything was generated on the fly th so Okay. Okay, yeah. Okay, so Yeah. Yeah, yeah, yeah. So could you say uh video name CINETIS slash something? Or not? You th okay. You think it would work, okay. Hmm. Mm even if you would r change a C_G_I_ for instance. Let's assume that we say we don't do any scene thing, then we could remove like for instance the scenes, right? I was thinking that here it says that you have online video structure correction. Uh which means that Okay. Yeah. Uh. This is wish wishable uh functions. Okay, to do this. Okay. Okay. Okay. Okay. No, I was thinking No, because I was thinking if it's there, I mean we could use it uh uh we could uh you see but uh yeah. No, no. Fro from outside we can view them. Yeah, from home I was able to access them. No. Okay. Nobody actually it has been downloaded several times and people there are some lawyers that are putting no. But yeah. Y Yeah yeah, yeah yeah, no no, yeah. No. But yeah, it's good to Yeah, if we i yeah. It might be good indeed if we start d building some experiments like this still to have a password. Right, yeah, yeah. Yeah yeah, yeah yeah. Hmm. Yeah, yeah what's, yeah, yeah. Hmm. Yeah, it was done indeed f to be more general than just to do the shot detection, so that's why the code is a bit more complex, I think, currently. Uh but uh Yeah. Yeah, yeah. Just Yeah. Ah okay, yeah yeah. Yeah, it was the same the same ones as the one you saw uh in my code, yeah. Yeah, yeah. Yeah. Yeah, yeah, yeah, yeah, yeah, definitely, yeah. Yeah, uh okay. Yeah, I think it's it's good, yeah. Okay, that's good, yeah. Yeah. Yeah, but why is after, I mean if there is a shot you don't know between because you dropped one frame, you don't know where is the sh uh the d the shot. Sliding window. Yeah, right. Yeah. I don't know, I'm I'm sure that they will be well I hope they will be a of something like this, yeah. The output. Yeah. This generates yeah. Ho how do you generate the Real Media, yeah. Okay. Oh o o, oh yeah, you mean the Java framework, uh what uh Mike presented. We cannot play uh Real Media, okay. Okay. Formats, formats, yeah. Okay, so I think this was good to see the the tool and uh tomorrow we are going to re-say most of the things, but shorter I f I believe, right, I mean it's bad to explain. And I think maybe later w yeah, we we can start doing simple Yeah, yeah, yeah. Yeah, the full uh Yeah, mi this might be interesting to see tomorrow. Of course maybe there was the ah, one hundred seventy shots, or it's a lot, but uh after I mean at this point we do we have nothing we can do unless they know what they want to do with it, right? Yeah, yeah. Yeah, but I think I think is it's too it's going very be very to be very good for them to visualise how you can organise things and maybe what they can do, how they and so this is important. Mm-hmm, yeah. Yeah, initially yeah. Well, we didn't because you tried to do something Okay. Yeah, like for instance if we need to find sub-shots, usually it it for instance is better to have the whole shot to decide on how to build the sub-shots. But it's true that maybe I if you store all the distance at that the distances we could do some basic stuff. Okay. Yeah yeah, yeah. Yeah yeah, yeah. Yeah, yeah. That's the thing, yeah. Ah, th the gap you mean or no? Oh no, you mean in the Bhattacharya distance. Yeah. Yeah, yeah. Yeah. Yeah. Yeah. Okay. So. It's okay.